Agile Sysadmin and the Art of Infrastructure Automation
I recently gave a “Devops’-flavoured talk at the London Ruby Users Group on agile sysadmin practices, agile web ops, configuration languages and infrastructure automation. The slides are below, and you can also see:
- Video of the talk (courtesy of our friends at SkillsMatter
- Audio podcast of the talk (ably captured by Murray Steele, as the soundtrack on the video is a little faint)
The idea of the talk was to look at modern sysadmin practices such as version control, configuration management and automation, and to show developers that with languages like Puppet, infrastructure is just code. “We’re all developers now,” as Julian Simpson (“The Build Doctor”) neatly put it. (Video of Julian’s talk here.)
The benefits of version control with tools like CVS, Git and Subversion are well-known in software development:
- You have a complete history of your project
- Separate teams and developers can have their own working copies of the code, and then re-integrate them for testing
- You can use branches to develop speculative or experimental features without affecting the main trunk
- You can use branches to maintain and patch old releases
- You can roll back to known good versions when things don’t work
- All changes are logged with metadata of who made the change, and most importantly, why
What’s becoming increasingly clear thanks to devops conferences such as DevOpsDays is that these benefits apply equally to the systems and infrastructure development that sysadmins do. In the bad old days, sysadmins would make changes directly on the server, which were often undocumented and irreversible. Now with properly managed version control and central repositories, sysadmins can exercise the same degree of control over their ‘code’ as the development team.
The idea of configuration management is that all the information that makes a server useful for some specific task - being a web server, for example, or a working database - is stored in some central place and automatically distributed to the server.
For example, all the work that a sysadmin might do manually to build up a newly provisioned Linux server, such as installing Apache, adding virtual host configuration, setting up firewall rules, installing PHP with a special config to support some application, and so on - all this can be brought under automatic management.
Puppet’s client-server architecture is ideal for managing configuration across multiple servers - with no theoretical upper limit to the number of machines that can be controlled this way. There are sites managing thousands of servers with Puppet!
Once you have version control and configuration management in place, it is possible to automate most of the tedious, repetitive tasks which consume sysadmins’ time and energy. For example, building new servers, application deployment, continuous integration, clustering, monitoring and backups can all be automated and managed centrally. Agile web ops means using automation to reduce the cost of changes to your existing infrastructure.
This has many benefits:
- Operations staff are happier
- You can provide resources more quickly to developers and business
- Your servers are more reliable and robust
- You have a change control process, which improves security and accountability
- You can scale to tens or hundreds of servers which would be impossible to set up and maintain by hand
The most important benefit for the sysadmin is that she can devote much more time to her other important job roles:
- Network architect
- Budget manager
- Performance engineer
- Security analyst
- Development liaison
- Teacher and team leader
Finally, because sysadmins are now developers, they are working alongside the other developers in the organisation on an equal footing, and with an equal interest in making development and deployment smooth, easy, reliable and fun. Programming skills and good engineering practices can spread throughout the technical team, and developers and sysadmins can come together with an attitude of respect, sharing and co-operation.
That’s good news for everybody!
Find out more
You can get up and running with Puppet in about 10 minutes with my Puppet tutorial series.
Other tools and resources
Agile sysadmin and consultant Patrick Debois has a very comprehensive review of the current state of the art in automated provisioning and configuration management. This is required reading for anyone interested in getting started with infrastructure automation: