Monthly Archives: November 2012

cfengine – some useful examples / or how I learn’t about the bomb and tried Puppet instead / salt?

Building on the initial post about cfengine we’re going to try out some things that may actually be useful.

My goal would be to make /etc/resolv.conf identical between all the machines.

The server setup is the lustre cluster we built in a previous post.

In this post you’ll first see two attempts at getting cfengine and then puppet to do my bidding until success was finally accomplished with salt.

Cfengine

Set up name resolution to be identical on all machines.

http://blog.normation.com/2011/03/21/why-we-use-cfengine-file-editing/

Thought about

Make oss1 and client1 not get the same promises.

Perhaps some kind of rule / IF-statement in the promise?

Cfengine feels archaic. Think editing named/bind configs are complicated? They are not even close to setting up basic promises in cfengine.

Puppet ->

http://puppetlabs.com/

CentOS 6 Puppet Install

vi /etc/yum.repos.d/puppet.repo
pdcp -w oss1,client1 /etc/yum.repos.d/puppet.repo /etc/yum.repos.d/puppet.repo

Sign certificates:

puppet cert list
puppet cert sign 
sudo puppet cert sign --all

For puppet there’s a dashboard. This sounds interesting. Perhaps I won’t have to write these .pp files which at a glancelooks scarily similar to the cfengine promises.

yum install puppet-dashboard mysqld

service start mysqld

set mysqld password

create databases (as in the database.yml file)

after this I didn’t get much further… But I did get the web-server up. Although it was quite empty…

salt

Easy startup instructions here for getting a parallel shell going:

After it’s set up you can run a bunch of built-in special commands, see the help section about modules.

salt ‘*’ sys.doc|less

will give you all the available modules you use :)

Want to use it for configuration management too? Check out the ‘states‘ section.

What looks bad with salt is that it’s a quite new (first release in 2011)

Salt is a very common word so it makes googling hard. Most hits tend to be about cryptography or cooking.

To distribute (once) the resolv.conf do you run this on the admin-server: salt-cp ‘*’ /etc/resolv.conf /etc/resolv.conf

On to states to make sure that the resolv.conf stays the same:

  1. uncomment the defaults in the master-file about file_roots and restart the salt-master service
  2. create /srv/salt and ln -s /etc/resolv.conf /srv/salt/resolv.conf
  3. create a /srv/salt/top.sls and a /srv/salt/resolver.sls

 

In top.sls put:

base:
 '*':
   - resolver

In resolver.sls put:

/etc/resolv.conf:
 file:
  - managed
  - source: salt://resolv.conf

Then run: salt ‘*’ salt.highstate

How to get this to run every now and then? Setting up a cronjob works.

Haven’t been able to find a built-in function to accomplish this but then again, all I’m doing here is scratching at the surface so it’s working and I’m happy :)

pdsh – parallell distributed shell

pdsh

This a software to run commands on a set of servers.

For example ‘pdsh -a uname -av’ will give you “uname -av” of all machines.

http://techsnail.com/howtos-tutorials/installing-pdsh-on-hpc-cluster/

It can be installed from rpmforge.0

wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
rpm -ivh rpm-release*
yum install pdsh

after that you can immediately run “pdsh -w oss1,client1 uname -av” to run a command on a remote node.

It’s possible to set up so that it executes on a pre-defined list. Check out /etc/machines.

Extremely useful if you want to save some time :)

cfengine – what’s that about?

http://cfengine.com/what-is-cfengine

It’s a (old) software that is used to make sure that (for example) the same config files are used on all machines. There are several other CMSs, for example puppet. Wikipedia has a nice overview of them.

Let’s use the lustre   machines we set up in a previous post.

On cfengine.com there are many examples too.

Inside a policy you have a promise.

Install

Installing on an RPM-based distribution is easy, cfengine has their own repository where the community edition is available.

http://cfengine.com/cfengine-linux-distros

Get the gpg-key, import it, set up the repository-file and install “cfengine-community”.

Check if “cfengine3” is set to start on boot.

Test

A small example how to write a promise.

  • “cf-promise -f ” can be used to test that a promise is valid (syntax and more is OK)
  • “cf-agent -f” run the promise, so if we use the example in the link above it echoes a Hello World.

 

Client/Server

Client pulls policies from the server.

policy-server: mds – 192.168.0.2
client1: client1 – 192.168.0.4
client2: oss1 – 192.168.0.3

on the policy-server hit: “/var/cfengine/bin/cf-agent –bootstrap –policy-server 192.168.0.2”

open port 5308 on the policy-server.

After you see “-> Bootstrap to 192.168.0.2 completed successfully” you can run the same cf-agent command on the client. This points it to use 192.168.0.2 as the policy-server.

No need to open port on the clients.

On the policy-server add this to /var/cfengine/masterfiles/cftest1.cf:

bundle agent test
{
 files:
  "/tmp/cf_test_file"
   comment => "Promise that a plain file exists with stated permissions",
    perms => mog("644", "root", "sys"),
   create => "true";
}

Then in /var/cfengine/masterfiles/promises.cf you can’t follow the guide verbatim, the promises.cf needs to look like this (really important to have “, ” as a separator between the bundles, notice the space after the “,”.

   body common control 
     {
     bundlesequence => { "main", "test" };
             inputs => { 
                       "cfengine_stdlib.cf", 
                       "cftest1.cf",
                       };
            version => "Community Promises.cf 1.0.0";
     }

After that you can run “cf-agent -Kv” on the client, and it will do what is promised in the cftest1.cf file!

Try to change ownership/permissions on the file, in a while it will have been changed back :)

In /var/cfengine/promise_summary.log you’ll see if it couldn’t keep a promise and if it corrected the mistake.

Distribute it

And to get oss1 the same file. Just run the good old “/var/cfengine/bin/cf-agent –bootstrap –policy-server 192.168.0.2” on it and eventually that file /tmp will pop up in there too. Nice!

Some useful stuff.

I’ll probably try out some more useful things in the near future.

Streamline resolv.conf settings, ip routes, config files for software like to make sure /etc/dcache/dcache.conf is the same on all pool servers or why not a kind of user database? Like for /etc/passwd? Check out the solutions on cfengine.com!