Automating Installations with Sunzi & Red Hat Subscription Manager

One of the things about being a Consultant in Red Hat or any other consulting company, is that sometimes cannot control how SO installations are performed. Sometimes all you get is a “naked” SO installation with default options selected and you need to work your way up from there. You can do automated installations using Kickstarts but needs a VM or server with no SO whatsoever.

What’s the problem with this? Not much if you are just installing one or two systems at a time. However, what happens when you need to do the same installation on more than 4 servers? Say 10? You get the idea.

There are plenty of server provisioning tools out there. You have the most known ones such as Puppet and Chef. Sure, those are great tools, but they need a centralized server and it’s most of all, to maintain configurations running after deployment. There’s also Capistrano, but it’s somewhat complex to setup, specially if we do not need all those options and we are not deploying applications, but rather installing and configuring packages.

So what I needed was a tool that could be easy to setup, written en either ruby or bash (shell), and most of all, simple.

Thank god I found sunzi, and, as the motto says, it’s a “Server provisioning utility for minimalists“. I have been playing with it for a week or so, and I must say, I absolutely love it!

Let me start by adding, this utility doesn’t need any centralized server, or complex configuration files. Here are some of the nice features it has:

  • It’s written in ruby and configuration files are pure shell (bash)
  • You can embed ruby code on top of the bash files for dynamic configuration (think of it as variables)
  • You can set “recipes” just like Chef and even set server “roles” (more on that later)
  • Does not require dependencies!! How cool is that?

So I started to see how can I integrate this tool in my projects. Turns out, it wasn’t that difficult. Let’s get started!

First of all, you need get the sunzi gem. Please note you will need a *nix based computer to do this. Windows will just NOT work. If you don’t have gem installed, then get at it. All instructions are for Mac/Fedora based Linux.

$ sudo yum install gem

Once this is installed you can just do:

$ gem install sunzi

This will install sunzi, and now all you need to do is to create a new “project” in sunzi and start working from there:

$ sunzi create

create sunzi/.gitignore
create sunzi/sunzi.yml
create sunzi/install.sh
create sunzi/recipes/sunzi.sh
create sunzi/roles/db.sh
create sunzi/roles/web.sh
create sunzi/files/.gitkeep

OK, now one of the most important files are sunzi.yml (where we put all our “dynamic” configuration) and install.sh where you can define custom commands or create bash functions. A quick look at the sunzi.yml file:

---

# Dynamic variables here will be compiled to individual files in compiled/attributes.
attributes:
environment: production
ruby_version: 2.1.2
postgres_user:
postgres_database:
postgres_password:
deploy_user:
app_name:
clustername:
rhn_user:
rhn_pass:
ipnode1:
ipnode2:
node1name: node1
node2name: node2
riccipass:

# Remote recipes here will be downloaded to compiled/recipes.
recipes:
# rvm: https://raw.github.com/kenn/sunzi-recipes/master/ruby/rvm.sh
# dotdeb: https://raw.github.com/kenn/sunzi-recipes/master/debian/dotdeb-wheezy.sh
# backports: https://raw.github.com/kenn/sunzi-recipes/master/debian/backports-wheezy.sh
# mongodb-10gen: https://raw.github.com/kenn/sunzi-recipes/master/debian/mongodb-10gen.sh

# Files specified here will be copied to compiled/files.
# files:
# - ~/.ssh/id_rsa.pub

# Fine tune how Sunzi should work.
preferences:
# Erase the generated folder on the server after deploy.
# Turn on when you are done with testing and ready for production use.
erase_remote_folder: true

# Skip retrieving remote recipes when local copies already exist. This setting helps
# iterative deploy testing considerably faster, when you have a lot of remote recipes.
cache_remote_recipes: true

# Evaluate files as ERB templates. When enabled, you can pass dynamic values in the form
# of <%= @attributes.environment %> in recipes, roles, files and install.sh.
eval_erb: true

As I’m defining roles for each server (inside the roles/ folder), in this file I’m setting up pretty much all variables that I will use for each one of my roles (or recipes).

Once I’m done with my variables, I insert those into my roles files, for example, in Red Hat, unless you have an active subscription, you will not be able to download any packages. So what I did was to insert the subscription manager inside the install.sh file which will be ran every time I do a deploy, it doesn’t matter if it’s in the form of a recipe or role:

#!/bin/bash
set -e

# Load base utility functions like sunzi.mute() and sunzi.install()
source recipes/sunzi.sh

# This line is necessary for automated provisioning for Debian/Ubuntu.
# Remove if you're not on Debian/Ubuntu.
#export DEBIAN_FRONTEND=noninteractive

# Add Dotdeb repository. Recommended if you're using Debian. See http://www.dotdeb.org/about/
# source recipes/dotdeb.sh
# source recipes/backports.sh

# Update apt catalog and upgrade installed packages
#sunzi.mute "apt-get update"
#sunzi.mute "apt-get -y upgrade"

# Register server against RHN
if subscription-manager version | grep -w 'not registered'; then
subscription-manager register --username=<%= @attributes.rhn_user %> --password=<%= @attributes.rhn_pass %>
subscription-manager attach --pool=xxxxxxxxxxxxxxxxxxxx

# We have to disable all repos that we will not be using:
yum-config-manager --disable \*

# Enable basic repo
yum-config-manager --enable rhel-6-server-rpms

# Install packages
yum -y install git ntp curl

else
echo 'This system is already registered and has repos available'
yum repolist
fi

# Install sysstat, then configure if this is a new install.
# if sunzi.install "sysstat"; then
# sed -i 's/ENABLED="false"/ENABLED="true"/' /etc/default/sysstat
# /etc/init.d/sysstat restart
# fi

# Set RAILS_ENV
# environment=<%= @attributes.environment %>

# if ! grep -Fq "RAILS_ENV" ~/.bash_profile; then
# echo 'Setting up RAILS_ENV...'
# echo "export RAILS_ENV=$environment" >> ~/.bash_profile
# source ~/.bash_profile
# fi

# Install Ruby using RVM
# source recipes/rvm.sh
# ruby_version=<%= @attributes.ruby_version %>

# if [[ "$(which ruby)" != /usr/local/rvm/rubies/ruby-$ruby_version* ]]; then
# echo "Installing ruby-$ruby_version"
# # Install dependencies using RVM autolibs - see https://blog.engineyard.com/2013/rvm-ruby-2-0
# rvm install --autolibs=3 $ruby_version
# rvm $ruby_version --default
# echo 'gem: --no-ri --no-rdoc' > ~/.gemrc

# # Install Bundler
# gem install bundler
#fi

Do you see the <%= @attributes.variable %> part? This is where once you deploy, it gets changed by the variables you set using the sunzi.yml file (a YAML file). As you can see, there’s nothing fancy here. I’m not a programmer, so maybe this file can be a lot more elegant, but for the time being, it serves my purpose.

Finally, what we need is to setup a role. A role is a series or commands that will get executed when invoking “sunzi deploy host role“. Here’s what I have for a High Availability host:

# Install High Availability

if rpm -qa 'cman' | grep -w 'cman'; then
 echo 'Cluster packages are already installed'
else
 yum-config-manager --enable rhel-ha-for-rhel-6-server-rpms
 yum -y groupinstall 'High Availability'
 echo <%= @attributes.riccipass %> | passwd --stdin ricci
 service ricci start
 chkconfig ricci on
 chkconfig cman on
 chkconfig rgmanager on
 chkconfig modclusterd on
fi

# We need to check if firewall is on and if so, load appropriate rules
IPT="$(/sbin/service iptables status)"
IPTEXPR="iptables: Firewall is not running."
if [ "$IPT" = "$IPTEXPR" ];then
 echo 'Firewall not running!'
else
# These are the iptables rules for cman
iptables -I INPUT -m state --state NEW -m multiport -p udp -s 192.168.1.0/24 -d 192.168.1.0/24 --dports 5404,5405 -j ACCEPT
iptables -I INPUT -m addrtype --dst-type MULTICAST -m state --state NEW -m multiport -p udp -s 192.168.1.0/24 --dports 5404,5405 -j ACCEPT

# These are the iptables rules for dlm
iptables -I INPUT -m state --state NEW -p tcp --dport 21064 -j ACCEPT

# These are the iptables rules for ricci
iptables -I INPUT -m state --state NEW -p tcp --dport 11111 -j ACCEPT

# These are the iptables rules for modclusterd
iptables -I INPUT -m state --state NEW -p tcp --dport 16851 -j ACCEPT

# These are the iptables rules for luci
iptables -I INPUT -m state --state NEW -p tcp --dport 8084 -j ACCEPT

# These are the iptables rules for igmp
iptables -I INPUT -p igmp -j ACCEPT

# Let's restart the firewall
service iptables save ; service iptables restart
fi

# Remove acpi support for correct fencing
chkconfig --del acpid

# We also need to disable NetworkManager if it's activated
chkconfig NetworkManager off
service NetworkManager stop

# Let's set the correct SELinux booleans in case it's running
if getenforce = 'Enforcing' /dev/null 2>&1; then
 setsebool fenced_can_network_connect 1
else
 echo 'SELinux is not running on this system!'
fi

# Enable basic cluster skeleton
mv files/cluster.conf /etc/cluster/cluster.conf
mv files/hosts /etc/hosts

Once again, not elegant, but it works. This file ensure all packages are installed, firewall is setup for HA and grabs files from the /files folder and puts it on the remote server (/etc/hosts for example).

The final step is to do the actual deploy. I like to copy my SSH key files to the server where I’m deploying. However, if you do not have this setup, it will ask for the root password:

$ ssh-id-copy root@remote.host

And now we invoke sunzi and it should start doing it’s magic. It should automatically register the server against the Red Hat Network, get the correct subscriptions, enable the repos and set everything else:

$ sunzi deploy remote.host rolename
identical compiled/attributes/environment
identical compiled/attributes/ruby_version
identical compiled/attributes/postgres_user
identical compiled/attributes/postgres_database
identical compiled/attributes/postgres_password
identical compiled/attributes/deploy_user
identical compiled/attributes/app_name
identical compiled/attributes/clustername
identical compiled/attributes/rhn_user
identical compiled/attributes/rhn_pass
identical compiled/attributes/ipnode1
identical compiled/attributes/ipnode2
identical compiled/attributes/node1name
identical compiled/attributes/node2name
identical compiled/attributes/riccipass
identical compiled/recipes/sunzi.sh
identical compiled/roles/resilstor.sh
identical compiled/roles/highavail.sh
identical compiled/roles/db.sh
identical compiled/roles/web.sh
identical compiled/roles/loadbalancer.sh
identical compiled/files/pg_hba.conf
identical compiled/files/hosts
identical compiled/files/cluster.conf
identical compiled/_install.sh
identical compiled/install.sh
Warning: Permanently added 'xx.xx.xxx.xxx' (RSA) to the list of known hosts.
server type: This system is currently not registered.
The system has been registered with id: 4739242dd-r23s-r32s-r32s1-r32cwfht56543
Successfully attached a subscription for: Red Hat Subscription
Loaded plugins: product-id============================== repo: rhel-source ===============================
[rhel-source]
bandwidth = 0
base_persistdir = /var/lib/yum/repos/x86_64/6Server
baseurl = .../en/os/SRPMS/
cache = 0
cachedir = /var/cache/yum/x86_64/6Server/rhel-source
cost = 1000
enabled = False
enablegroups = True
exclude =
failovermethod = priority
gpgcadir = /var/lib/yum/repos/x86_64/6Server/rhel-source/gpgcadir
gpgcakey =
gpgcheck = True
gpgdir = /var/lib/yum/repos/x86_64/6Server/rhel-source/gpgdir
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release<span style="font-size: 1em; line-height: 1.5;">...

 

If it was all OK, then you should have a server ready with all packages in place and even configuration files. Maybe you will need to tweak it a little bit more however, you get the idea from this.

There are some recipes in the main sunzi repo, and also if you want to take a look at mines, they are in my GitHub account.

I think that should be all for now. Let me know if it works for you!

Tadá!

Recent Posts

Recent Comments

    Meta

    tomas Written by: