At LifeChurch.tv, we are building a new platform for managing our Linux servers, and key to our strategy is the use of puppet. The nice thing about puppet is that you can define certain policies for your servers, and puppet will go through the process of bringing your member servers (nodes) into compliance.

This means you can setup your infrastructure by describing it. Describe what you want in a configuration file, and puppet ensures that your nodes match your description continuously. It even works across distributions and OSes, though this is not as relevant for our environment because we will be running entirely on Debian.

Puppet is not the only tool that does this. Other similar tools include cfengine, Chef, and Capistrano.

Puppet Setup

There are some really nice guides on how to setup puppet. One excellent quick-start guide is on the Debian Administration site. Follow that and you will have a working puppet network. The only thing missing are the actual rules you want to enforce, and so let’s talk about that!

Why Use Puppet to Manage User Accounts? (and not NIS, LDAP, etc)

One of the benefits to managing user accounts in puppet is the fact that it is decentralized. Each user account is just a normal user account on the managed server. There is nothing special about the user accounts puppet creates other than the fact they were created by puppet and not by a human administrator.

The nice thing about this is that if the main host dies, we do not lose authentication. Which means that our puppetmaster server (or NIS/LDAP server) need not have any special uptime requirements. If an emergency happens, we can focus on getting our production servers up, and focus on getting the puppetmaster up on an “as needed” basis.

The downside to this is that puppet is not necessarily really designed to manage “normal” login user accounts (as opposed to system accounts). The biggest way this comes up is that, although you can set the password in puppet, puppet continually monitors system settings (good) and if it notices that the password has changed, will reset it. (bad)

I do not want to monitor user passwords on our network, so there needs to be a way to set a password and have puppet stop monitoring this password. Fortunately, once you figure out the trick, this is actually really quite easy. But first, let’s get some definitions out of the way.

User Setup Definition

So I created a file in /etc/puppet/manifests/classes/user.pp which will hold all the user definition rules. Here’s what our add_user definition says:

define add_user ( $email, $uid ) {

            $username = $title

            user { $username:
                    comment => "$email",
                    home    => "/home/$username",
                    shell   => "/bin/bash",
                    uid     => $uid
            }

            group { $username:
                    gid     => $uid,
                    require => user[$username]
            }

            file { "/home/$username/":
                    ensure  => directory,
                    owner   => $username,
                    group   => $username,
                    mode    => 750,
                    require => [ user[$username], group[$username] ]
            }

            file { "/home/$username/.ssh":
                    ensure  => directory,
                    owner   => $username,
                    group   => $username,
                    mode    => 700,
                    require => file["/home/$username/"]
            }

            exec { "/narnia/tools/setuserpassword.sh $username":
                    path            => "/bin:/usr/bin",
                    refreshonly     => true,
                    subscribe       => user[$username],
                    unless          => "cat /etc/shadow | grep $username| cut -f 2 -d : | grep -v '!'"
            }

            # now make sure that the ssh key authorized files is around
            file { "/home/$username/.ssh/authorized_keys":
                    ensure  => present,
                    owner   => $username,
                    group   => $username,
                    mode    => 600,
                    require => file["/home/$username/"]
            }
    }

The define is like defining a function. And each statement in the definition is a resource that puppet can manage. (list of resources and their options) The first user statement creates the actual user with all the passed variables. I like to specify the UID so they can be consistent across nodes. Next statement creates a group of the same name and id as the user. We want the group to be created after the user, so we have a require statement to setup this dependency.

Next we setup the home folder and ~user/.ssh folder so that we can manage ssh keys as well. Then we have the exec statement which executes a script I created which gives the user a random password and emails them to let them know what it is. Pretty simple. It is in Narnia, which is our custom toolkit that is synced to each server. (setup using another puppet rule… for another blog post)

Now, normally puppet will run this script at every refresh which means that users would have their passwords reset every 30 minutes. Not good. So there are three key lines:

refreshonly: this will only execute this rule on “refresh”–whenever the subscribed event changes. In our case, this should only happen when the user is created.

subscribe: the event we’re interested in. In our case, the creation of the user.

unless: I am not sure if this is completely necessary in our case, but this prevents the rule from executing if a certain condition returns true. So I wrote a quick one liner that opens the shadow file, finds the appropriate user, and returns the password hash if it does not contain an exclamation mark. This is key because when puppet creates the user, it creates it with a blank password which shows up as a ! in the shadow file. This one liner will not return that, allowing the reset script to run and to set a password. But if a password has already been set (there is no !), this returns true and prevents puppet from resetting the password. This may not be necessary, but acts as a failsafe in case I do not properly understand how refreshonly and subscribe work.

The nice thing about resetting a password with a custom script is that even if things go awry and this script is executed at the wrong times, we can add error checking in the script itself.

So that is how users are setup and their passwords managed. How about ssh keys?

SSH Key Management

If you don’t know about them… SSH keys are awesome. It uses asymmetric public-key encryption, which I do not understand the mathematics of, but basically you get two big numbers that are somehow mathematically related. They are related in such a way that you can generate the public key if you know the private key, but it’s impossible to get the private key if you only know the public key.

Crazy, right? I don’t know how that works either. But the cool thing is that it enables password-less logins. If I setup my SSH keys, I can login to servers all over the world without entering a password. And it’s still more secure than using passwords.

Anyway. I want to be able to take advantage of these properties, and I want to be able to use puppet to manage it. So in the same users.pp file as above, I have some definitions related to ssh keys:

define add_ssh_key( $key, $type ) {

            $username       = $title

            ssh_authorized_key{ "${username}_${key}":
                    ensure  => present,
                    key     => $key,
                    type    => $type,
                    user    => $username,
                    require => file["/home/$username/.ssh/authorized_keys"]

            }

    }

ssh_authorized_keys is a built-in type in puppet, which just means that puppet has a bunch of code to handle everything we need, we just have to tell it what to do.

So we create an authorized key, and we name it: [Username of user we’re logging in as]_[Full Text of Key]. In many types, the name is important and used somewhere, but not in ssh_authorized_keys. In this case, puppet just wants a unique name, and this is an easy way to give it a unique name. (since it doesn’t make sense to have puppet manage the same key multiple times for one user)

The key things here (ha!) are that we need to give it the actual key, the key type (ssh_rsa or ssh_dsa?), and the user we want to make this an authorized key for. In our case, I want to make sure that this runs only after the authorized_keys file is created. This is probably not strictly necessary.

I have an identical function for root keys, it just checks to ensure that there is a rule for creating the /root/authorized_keys file. We can’t use the definition above, because the root home folder is /root, not /home/root/.

Putting It All Together!

Nothing we’ve done so far will do anything on any of the nodes. They’re just Lego building blocks. Now let’s put them together and build something cool. Like a giant Statue of Liberty. Or not.

The cornerstone for any puppet configuration rests in the /etc/puppet/manifests/site.pp file. So first of all, we want to make sure our /etc/puppet/manifests/classes/*.pp classes get included:

import "classes/*.pp"

Then we need to specify node rules. In our case, I just want a default rule that applies to all nodes. (node default)

node default {

            add_user { rrunner:
                    email    => "road.runner@acme.com",
                    uid      => 5001
            }

We call the add_user function we defined waaaaay up there like 200 paragraphs ago. In this case, the name is important. It becomes the username. Then we add their email address (which goes in the gecos/comment field in the /etc/passwd file). This isn’t strictly necessary, but we do need a way to get the user’s email so we can send them their randomly generated password, so we just need to include it.

Next, we want to add an ssh key for road runner:

add_ssh_key { rrunner:
                    key     => "AAAAB3NzaC1yc2EAAAABIwAAAQEAwtnN3Nmkn8WKfBUs4/AwCmthfsY6TXmEe63d2Okeo3QtpUvceXj83bVqerK6h62bEGb7LtE2oW8utiE8ZlWmeViMdIZo6OQVOMj69HgPZu3IKSIYW5hTVWhb5FmQOOtGk5m1uxJyeBI5ivmVJtQIrH6gOkoOP1X23PqLCUnb1Wur9J6NCAOOLtJQEl+CMTSRqNZ6VU/4Kvu0FxSiAqHdi5i4zpob3HIWXSC0Ugh664jqvjjJI7ZLuC4Ym3BFK+uZKVX3yKIx0sbiZm+IMBvuUJzmpfPTMPrMyZuq7FxSUjIv+TX4XKwxv8ysU39k1WllOYT5kDwkOnJ5NLt4zqJQVQ==",
                    type    => "ssh-rsa"

            }

    }

And… that’s it. Save and at the next puppet refresh, puppet will generate user accounts for road runner on each node, include road runner’s public key, and email road runner that his account is ready.

I hope this has been helpful. If you have any questions on our setup, please leave a comment. I setup a github project with the example configuration I used in this post. It also includes more explanatory comments.

Meep meep.

Helpful Resources

• Puppet configuration on github
• Docs at Puppet Labs
• Native Resource Types
• Installing and Using puppet (part 2)
• Puppet Best Practices
• The best kind of “puppet”