Configure A Server With Chef-Solo In Five Minutes

Note: Today, I recommend starting with Ansible instead of Chef Solo. It is far easier to get started with. See: Start Learning Ansible With One Line And No Files.

The Chef Fast Start Guide has been renamed and rewritten since I first wrote this in 2013. It does seem better, but it still looks anything but “fast”.

Chef is an open-source tool for automating the configuration of servers. It, or its cousin Puppet, is a standard tool of the cloud operations engineer.

Unfortunately, if my Twitter feed is any guide, the world is full of people who could be using Chef, but think it’s too complicated. And who can blame them? When I encountered it in 2013, the Chef “Fast Start Guide” did not inspire confidence. You’re probably thinking of using Chef to replace a straightforward shell script full of lines like this:

$ sudo useradd guest
$ sudo aptitude install emacs-nox

But before telling you how to do any of that, Chef’s “fast” start guide instructed you to:

  • Register a user on the Hosted Chef paid service (requiring a signup form, a credit card, and three click-through legal agreements) in order to
  • download a pile of cryptography keys, to
  • use when installing Git (!) and setting up a local repository, to
  • be managed by the knife tool, which
  • mediates between Hosted Chef, your local repository, and repos containing third-party ‘cookbooks’, which
  • contain ‘recipes’, which
  • according to their README files, probably install the Linux package you’re looking for.

This complexity isn’t pointless - if you play the devops game for a few months, you’ll discover why these features are there. But you may not need all this yet. Today, you’re just trying to replace one shell script.

Wouldn’t it be nice if there were a much simpler version of Chef? The good news is that Chef contains a simpler version of itself, known as “Chef Solo”. Chef Solo can manage basic configuration tasks, and it’s a lot easier to understand.

The bad news is that Chef Solo is still hard for a beginner to configure, because the instructions are all mixed up with the other Chef docs. So here’s a step-by-step guide.

Step One: Launch An Example Server

The best way to practice Chef (and most other tools) is to use a server running “in the cloud”. They’re cheap to rent, can be thrown away when you break them, and are much more consistent - you don’t need customized instructions for your particular operating system and hardware, for example, because you can just use the same standard Linux server as everyone else.

Let’s launch an “instance” on Amazon Web Services. If you’ve never signed up for AWS, click “Sign Up” on its home page, then launch an EC2 instance based on Ubuntu - the “Quick Launch Wizard” is the easiest to use. I used Ubuntu 12.04 LTS on a Micro-flavored instance; you don’t need anything larger for practice.

Step Two: Install Chef On The Example Server

Once you’re logged in to your new instance, you should begin by updating the already-installed Ubuntu packages:

$ sudo aptitude update
$ sudo aptitude safe-upgrade

If you are just practicing and in a big hurry you can skip the safe-upgrade, which takes more time than everything else in this guide. But even with throwaway instances I have a habit of upgrading the packages immediately. Security patches do you no good if you aren’t running them.

Then we install Chef and its prerequisites:

$ sudo aptitude install -y ruby ruby1.8-dev build-essential wget libruby1.8 rubygems
$ sudo gem update --no-rdoc --no-ri
$ sudo gem install ohai chef --no-rdoc --no-ri

Step Three: Create The Simplest Chef Configuration

When you run chef-solo on a machine, it will start configuring various things.

  • We tell it what to configure by providing a run list in a node.json file.
  • We tell it how to configure the run list by providing recipes, which are files full of commands (technically written in Ruby, but you don’t need to know Ruby to build one). Each recipe is kept in a collection called a cookbook (Chef Solo is fond of silly cooking metaphors) which in turn are kept in a cookbooks directory.
  • We use a solo.rb file to tell chef-solo how to find the cookbooks and the run list.

We’ll keep things tidy by putting all of these files inside our own ~/chef directory. Then that directory can be version-controlled, archived and distributed with tar, and so on.

Let’s practice by having Chef do something simple: Create a /tmp/helloworld.txt file. To teach Chef to do this we need a recipe, which we’ll call helloworld, and we’ll put that recipe inside a cookbook which is also named helloworld.

$ mkdir -p ~/chef/cookbooks/helloworld/recipes
$ echo '
file "/tmp/helloworld.txt" do
  owner "ubuntu"
  group "ubuntu"
  mode 00544
  action :create
  content "Hello, Implementor!"
end' > ~/chef/cookbooks/helloworld/recipes/default.rb

Next we’ll write a node.json to tell Chef to run the helloworld recipe:

$ echo '
  "run_list": [ "recipe[helloworld]" ]
}' > ~/chef/node.json

Finally we’ll tell Chef where to find the files we just created, using a solo.rb file.

$ echo '
  file_cache_path "/home/ubuntu/chef"
  cookbook_path "/home/ubuntu/chef/cookbooks"
  json_attribs "/home/ubuntu/chef/node.json"
' > ~/chef/solo.rb

Now we’re ready to run chef-solo. If we try doing so as our non-root ubuntu user we’ll get an error, so make sure to use sudo to run chef-solo as root:

$ sudo chef-solo -c ~/chef/solo.rb

We use the -c option to explicitly tell chef-solo where to read its configuration.

You should see a neat stream of messages from Chef as it works:

$ sudo chef-solo -c ~/chef/solo.rb 
[2013-01-02T17:51:09+00:00] INFO: *** Chef 10.16.4 ***
[2013-01-02T17:51:10+00:00] INFO: Setting the run_list to ["recipe[helloworld]"] from JSON
[2013-01-02T17:51:10+00:00] INFO: Run List is [recipe[helloworld]]
[2013-01-02T17:51:10+00:00] INFO: Run List expands to [helloworld]
[2013-01-02T17:51:10+00:00] INFO: Starting Chef Run for ip-10-203-63-20.ec2.internal
[2013-01-02T17:51:10+00:00] INFO: Running start handlers
[2013-01-02T17:51:10+00:00] INFO: Start handlers complete.
[2013-01-02T17:51:10+00:00] INFO: Processing file[/tmp/helloworld.txt] action create (helloworld::default line 2)
[2013-01-02T17:51:10+00:00] INFO: entered create
[2013-01-02T17:51:10+00:00] INFO: file[/tmp/helloworld.txt] owner changed to 1000
[2013-01-02T17:51:10+00:00] INFO: file[/tmp/helloworld.txt] group changed to 1000
[2013-01-02T17:51:10+00:00] INFO: file[/tmp/helloworld.txt] mode changed to 544
[2013-01-02T17:51:10+00:00] INFO: file[/tmp/helloworld.txt] created file /tmp/helloworld.txt
[2013-01-02T17:51:10+00:00] INFO: Chef Run complete in 0.191596 seconds
[2013-01-02T17:51:10+00:00] INFO: Running report handlers
[2013-01-02T17:51:10+00:00] INFO: Report handlers complete

if you run cat /tmp/helloworld.txt you should see a tiny greeting. Success!

Step Four: Add More Steps To The Recipe

You probably want to do something more exciting with Chef than build helloworld.txt files. But now you know almost everything you need for writing recipes using Chef’s built-in resources. We just used the file resource, and there’s a list of additional ones on the Chef Resources docs page.

Just pick another resource and try it out. For example, let’s have Chef create some cron jobs. Crontab editing is one of those tasks that make Chef worthwhile: Writing a custom script that lets you add, modify, or delete arbitrary crontab entries is a pain, but Chef has done the hard work already:

$ mkdir -p ~/chef/cookbooks/crondemo/recipes
$ echo '
  cron "log something" do
    action :create
    hour "*"
    minute "*"
    command "logger Hello!"
' > ~/chef/cookbooks/crondemo/recipes/default.rb

Don’t forget to edit the run_list in node.json to add recipe[crondemo]:

run_list": [ "recipe[helloworld]", "recipe[crondemo]" ]

Run Chef again:

$ sudo chef-solo -c ~/chef/solo.rb

Then run sudo tail -f /tmp/dates and wait for your new cron job to log a Hello!.

You can put more than one recipe in a cookbook – just create another file in its recipes directory:

$ echo '
  cron "log something else" do
    action :create
    hour "*"
    minute "*"
    command "logger Goodbye!"
' > ~/chef/cookbooks/crondemo/recipes/goodbye.rb

Edit node.json one more time:

"run_list": [ "recipe[helloworld]", "recipe[crondemo]", "recipe[crondemo::goodbye]" ]

Run Chef and your syslog will fill up with Hello! as well as Goodbye! messages.

When you get tired of these cron jobs you can use Chef to remove them from your system:

$ echo '
  cron "log something" do
    action :delete
  cron "log something else" do
    action :delete
' > ~/chef/cookbooks/crondemo/recipes/remove.rb

Edit node.json to replace the crondemo recipes in the run_list with crondemo::remove:

"run_list": [ "recipe[helloworld]", "recipe[crondemo::remove]" ]

Run chef-solo and your cron jobs should be gone. Use sudo crontab -l to prove that they’re gone.

Step Five: Export

Now that we have a ~/chef directory containing basic Chef configuration, we can tinker with the contents until we have the recipes we want for setting up our server. Then we can tar up the whole thing, dump it onto a newly launched instance, install Chef Solo and run it.

Alternatively, you can use Git to distribute the configuration. I put the example configuration up on Github. Launch a new Ubuntu instance and install Chef:

$ sudo aptitude update
$ sudo aptitude safe-upgrade
$ sudo aptitude install -y ruby ruby1.8-dev build-essential wget libruby1.8 rubygems
$ sudo gem update --no-rdoc --no-ri
$ sudo gem install ohai chef --no-rdoc --no-ri

(For future reference, these are the same install commands used by Chef’s official knife bootstrap tool - learn to use knife bootstrap and you can avoid typing all of this.)

Then do:

$ sudo aptitude install -y git
$ git clone chef
$ sudo chef-solo -c /home/ubuntu/chef/solo.rb

Now you can tinker with the recipes, test them, commit to the Git repository, and push that repository to your own Github account.

Now What?

  • Learn more Chef resources so that you can, say, install packages or restart services.

  • Learn how to download Chef community cookbooks into your /cookbooks directory, so that you don’t have to reinvent the recipes for doing complex things like bringing up mysql.