Skip navigation.

Puppet Beginner's Guide, 2nd Edition

Pre-order your copy now

A new edition of the bestselling Puppet Beginner’s Guide is now in preparation, for publication in May 2017, covering all the latest features of Puppet 4. Learn Puppet from scratch, from installation to managing complex infrastructures, including brand new sections on managing Amazon AWS cloud resources and Docker containers with Puppet. Every code example is available right now online in a special GitHub repo just for this book. There is also a complete, working Puppet infrastructure for you to download, adapt, and use on your own servers.

Audience

This book is designed for those new to Puppet, including system administrators and developers who are looking to manage computer server systems for configuration management. No prior programming or system administration experience is assumed.

Mission

The Puppet Beginner’s Guide gets you up and running with Puppet straight away with real-world examples. It will take you from complete beginner to confident Puppet user with a series of clear examples and case studies. By reading this book, you will learn to manage every aspect of your server setup. You will gain expert understanding of Puppet’s latest and most advanced features and put them to work.

Objectives and achievements

This book helps you to:

  • Install and set up Puppet and discover the latest and most advanced features
  • Automate your IT infrastructure
  • Use the latest features in Puppet 4+ and official modules
  • Manage clouds, containers, and orchestration
  • Learn best practices to make Puppet more reliable and high performing

General structure

  1. Getting started with Puppet
  2. Creating your first manifests
  3. Managing Puppet code with Git
  4. Understanding Puppet resources
  5. Variables, expressions, and facts
  6. Managing data with Hiera
  7. Mastering modules
  8. Classes, roles, and profiles
  9. Managing files with templates
  10. Controlling containers
  11. Orchestrating cloud instances
  12. Putting it all together

Detailed outline

About this book

A brief introduction to the aims and methods of the book, and how to use it.

1. Getting started with Puppet

In this chapter you’ll learn about some of the challenges of managing configuration on servers, some common solutions to this problem, and how automation tools such as Puppet can help. You’ll also learn how to download the GitHub repository containing all the source code and examples in this book, how to set up your own Vagrant virtual machine to run the code, and how to download and install Puppet.

  • Why do we need Puppet anyway?
    • Keeping configuration synchronized
    • Repeating changes across many servers
    • Self-updating documentation
    • Why not just write shell scripts?
    • Why not just use containers?
    • Why not just use serverless?
    • Configuration management tools
  • What is Puppet?
    • Resources and attributes
    • Puppet architectures
  • Getting ready for Puppet
    • Getting the Git repo
    • Installing Virtualbox and Vagrant
    • Running your Vagrant VM
    • Troubleshooting Vagrant
    • Installing Puppet manually

2. Creating your first manifests

The Puppet manifest is the complete expression, in code form, of how your server should be configured. It consists of a set of resources, each of which models some aspect of system configuration, such as a file, or a software package.

In this chapter you’ll learn how to write your first manifest with Puppet, and how to put Puppet to work configuring a server. You’ll also understand how Puppet reads and applies a manifest. You’ll see how to use Puppet to manage the contents of files, how to install packages, and how to control services.

  • Hello, Puppet: your first Puppet manifest
    • Understanding the code
    • Modifying existing files
    • How Puppet applies the manifest
    • Exercise
  • Managing packages
    • How Puppet applies the manifest
    • Exercise
  • Querying resources with puppet resource
  • Services
    • Getting help on resources with puppet describe
    • The package-file-service pattern
    • Understanding the code
    • Notifying a linked resource
    • Resource ordering with require

3. Managing Puppet code with Git

In this chapter we’ll install Git, create a repo, and import the manifests created in previous chapters. Readers will learn to make some changes to the manifests and commit them to the repo. Readers will learn how to check the revision history of the repo. We will clone our existing repo to a master repo, and deploy it on a new server. We will set up a script and cron job to pull changes from the master repo and apply them automatically.

  • What is version control?
    • Tracking changes
    • Sharing code
  • Time for action: creating a Git repo
    • Making your first commit
    • How often should I commit?
    • Branching
  • Distributing Puppet manifests
    • Creating a GitHub account and project
    • Pushing your repo to GitHub
    • Cloning the repo
    • Installing your manifests
  • Fetching and applying changes automatically
    • Writing a manifest to set up regular Puppet runs
    • Applying the run-puppet manifest
    • The run-puppet script
    • Exercise
    • Managing multiple machines

4. Understanding Puppet resources

We’ve already met three important types of Puppet resource: packages, files, and services. In this chapter we’ll go into a little more detail on these to see how to exploit their power to the full. We’ll also learn about three other important resource types: user, cron, and exec resources. Along the way we’ll talk about:

  • Files
    • Managing whole files
    • Ownership
    • Permissions
    • Directories
    • Trees of files
    • Symbolic links
  • Packages
    • Uninstalling packages
    • Installing specific versions
    • Installing the latest version
    • Installing Ruby gems
    • Installing gems in Puppet’s context
  • Services
    • The hasstatus attribute
    • The pattern attribute
    • The hasrestart and restart attributes
  • Users
    • Creating users
    • Understanding the user resource
    • Managing SSH keys
    • Removing users
  • Cron resources
    • Attributes of the cron resource
    • Randomizing cron jobs
    • Removing cron jobs
  • Exec resources
    • Automating manual interaction
    • Attributes of the exec resource
    • The user attribute
    • The onlyif and unless attributes
    • The refreshonly attribute
    • The logoutput attribute
    • The timeout attribute
    • How not to misuse exec resources

5. Variables, expressions, and facts

In this chapter you will learn about variables in Puppet, and discover the different types of data structures and data types that Puppet provides. You will find out how to use variables and values in expressions, and how to apply different resources depending on the value of expressions, using conditional statements. You will also learn how Puppet manifests can get data about the server using Facter, find out which are the most important standard facts, and see how to create your own custom facts. Finally, you will see how to use the each function to loop over arrays and hashes.

  • Variables
    • Booleans
    • Interpolating variables in strings
    • Arrays
    • Arrays of resources
    • Hashes
    • Setting resource attributes from a hash
  • Expressions
    • Comparison operators
    • Regular expressions
    • Conditional expressions
    • If statements
    • Case statements
  • Facts
    • The facts hash
    • The facter command
    • Hashes of facts
    • Facts in expressions
    • Memory facts
    • Networking facts
    • External facts
    • Executable facts
  • Iteration
    • The each function
    • Iterating over hashes

6. Managing data with Hiera

In this chapter you will learn why it’s useful to separate program logic from configuration data. You will see how to use Puppet’s built-in Hiera mechanism to store and query config data, including encrypted secrets such as passwords and SSL keys. You will also learn how to use Hiera data to construct Puppet resources directly.

  • Why Hiera?
    • Data needs to be maintained
    • Settings depend on servers
    • Operating systems differ
    • The Hiera way
  • Setting up Hiera
  • Querying Hiera
  • Writing Hiera data
    • File header
    • Single values
    • Arrays
    • Hashes
    • Interpolation
  • The hierarchy
  • Adding Hiera data to your Puppet repo
    • Troubleshooting Hiera
    • What belongs in Hiera?
  • Creating resources from Hiera data
    • Building resources from Hiera arrays
    • Building resources from Hiera hashes
  • Managing secret data
    • Setting up GnuPG
    • Setting up hiera-eyaml-gpg
    • Creating an encrypted secret
    • How Hiera decrypts secrets
    • Editing or adding encrypted secrets
    • Distributing the decryption key

7. Mastering modules

In this chapter you will learn about Puppet Forge, the public repository for Puppet modules, and you’ll see how to install and use third-party modules from Puppet Forge, using three of the most popular module management tools: puppet-module, librarian-puppet, and R10K. You’ll work through examples of using three key Forge modules: puppetlabs-apache, puppetlabs-mysql, and puppet-archive. Finally, you’ll learn how to develop your own Puppet module from scratch, working through a complete example, how to add appropriate metadata for your module, and how to upload it to Puppet Forge.

  • Using Puppet Forge modules
    • What is the Puppet Forge?
    • Finding the module you need
    • Using puppet-module
    • Using librarian-puppet
    • Using R10K
    • Managing dependencies with R10K
  • Using modules in your manifests
    • Using puppetlabs-mysql
    • Using puppetlabs-apache
    • Using puppet-archive
  • The standard library
    • ensure_packages
    • file_line
    • Other useful functions
  • Writing your own modules
    • Creating a repo for your module
    • Writing the module code
    • Creating and validating the module metadata
    • Installing your module
    • Applying your module
    • More complex modules
    • Uploading modules to the Puppet Forge

8. Classes, roles, and profiles

In this chapter you will explore the details of Puppet classes, the distinction between defining a class and creating instances of that class, and how to pass parameters to classes. You’ll learn how to create defined resource types, and how they differ from classes. You’ll also see how to organize your Puppet code using the concepts of nodes, roles, and profiles.

  • Classes
    • The class keyword
    • Passing parameters to classes
  • Parameter data types
    • Available data types
    • Range parameters
    • Content type parameters
    • Flexible data types
  • Defined resource types
  • Nodes, roles, and profiles
    • Nodes
    • Roles
    • Profiles

9. Managing files with templates

In this chapter we’ll learn about an important and powerful feature of Puppet: the template. We’ll see how to use a simple template to interpolate the values of Puppet variables, facts, and Hiera data into a file, and we’ll also introduce more complex templates using iteration and conditional statements to generate dynamic configuration files.

  • What are templates?
  • Using templates in your manifests
    • Referencing template files
    • Inline templates
    • Template tags
    • Computations in templates
    • Conditional statements in templates
  • Iteration in templates
    • Iterating over Facter data
    • Iterating over structured facts
    • Iterating over Hiera data
  • Working with templates
    • Passing parameters to templates
    • Validating template syntax
    • Rendering templates on the command line
  • Legacy ERB templates

10. Controlling containers

In this chapter we’ll look at the emerging topic of containers and see how it relates to configuration management. We’ll see how to use Puppet to manage Docker itself, as well as images and containers, and explore some different strategies for managing configuration within containers.

  • Understanding containers
    • The deployment problem
    • Options for deployment
    • Introducing the container
    • What Docker does for containers
    • Deployment with Docker
    • Managing containers with Puppet
  • Managing Docker with Puppet
    • Installing Docker
    • Running a Docker container
    • Running multiple instances of a container
  • Managing Docker images
    • Building images from Dockerfiles
    • Managing Dockerfiles
    • Rebuilding and updating containers
  • Building dynamic containers
    • Configuring containers with templates
    • Self-configuring containers
  • Persistent storage for containers
    • Host-mounted volumes
    • Docker volumes
  • Networking and orchestration
    • Connecting containers
    • Running Puppet inside containers
    • Container orchestration

11. Orchestrating cloud resources

In this chapter we will learn how to use Puppet to create and manage AWS cloud resources, such as EC2 instances.

  • Introducing the cloud
    • Automating cloud provisioning
    • CloudFormation
    • Terraform
  • Managing AWS cloud resources with Puppet
    • Setting up an Amazon AWS account
    • Generating AWS access keys
    • Installing the AWS SDK and credentials
    • Creating a key pair
    • Installing the puppetlabs-aws module
    • Choosing an Amazon Machine Image (AMI)
    • Creating an EC2 instance
    • Accessing your EC2 instance
    • The ec2_securitygroup resource
    • The ec2_instance resource
    • Creating a VPC and subnet
    • The ec2_vpc resource
    • The ec2_vpc_internet_gateway resource
    • The ec2_vpc_routetable resource
    • The ec2_vpc_subnet resource
    • Provisioning AWS resources from Hiera data
    • Other AWS resources
    • Orchestrator servers
  • Other cloud providers
    • Google Compute Engine
    • Microsoft Azure
    • DigitalOcean

12. Putting it all together

In this chapter we will apply all the lessons from previous chapters to see what a complete, working infrastructure looks like. This is a complete worked example which readers can use as the basis of their own Puppet codebase, adapting and expanding it as needed.

  • Getting the demo repo
  • Understanding the demo repo
    • The control repo
    • Module management
    • Nodes, roles, and profiles
    • Users and access control
    • SSH configuration
    • Sudoers configuration
    • Timezone and clock synchronization
    • Puppet configuration
  • The bootstrap process
  • Adapting the repo for your own use
    • Copying the repo
    • Configuring users
    • Adding node and role declarations
    • Modifying the bootstrap credentials
    • Bootstrapping a new server

Author Bio

John Arundel is a devops consultant, which means he helps people build world-class web operations teams and infrastructure, and has fun doing it. He was formerly a senior operations engineer at global telco Verizon, designing resilient, high-performance infrastructures for major corporations like Ford, McDonald’s, and Bank of America. He is now an independent consultant, working closely with selected clients to deliver web-scale performance and enterprise-grade resilience on a startup budget.

He likes writing books, especially about Puppet (The Puppet Cookbook is available from the same publisher). It seems that at least some people enjoy reading them, or maybe they just like the pictures. He also provides training and coaching on Puppet and devops, which it turns out is far harder than simply doing the work himself.

Off the clock, he is a medal-winning competitive rifle and pistol shooter, and a decidedly uncompetitive piano player. He lives in a small cottage in Cornwall, England and believes, like Cicero, that if you have a garden and a library, then you have everything you need.

You may like to follow him on Twitter at @bitfield.

Thanks are due to Luke Kanies, who created a configuration management tool that sucks less, and my many other friends at Puppet, including the invaluable Gareth Rushgrove, Adrien Thebo, and Gary Larizza. Many of the key ideas in this book came from them and others including Przemyslaw ‘SoboL’ Sobieski, Peter Bleeck, and Igor Galić. The techniques and examples in the book come largely from real production codebases, of my consulting clients and others, and were developed with the indispensable assistance of my friends and colleagues Jon Larkowski, Justin Domingus, Walter Smith, Ian Shaw, and Mike Thomas. Special thanks are also due to the Perseids Project at Tufts University and Bridget Almas, who patiently read and tested everything in the book, and made many important improvements; she suffered so that you don’t have to.