Posts tagged puppet

ansible introduction

posted on 2017-04-29 10:20

prerequisites for the next section

You know how to use ssh,ssh-keygen, ssh-copy-id so you can achieve your desired destination server of choice with passwordless keyauth for user root.

toying with ansible commandline, first needed notations

This is just for getting singe action run on single host or a group of hosts.

  • without using a playbook (= ansible's name for a detailed blueprint for what you want to install)
  • showing modules (module = ansible part implementing some function)
  • showing how to group contents with a hosts inventory file
  • why idempotence really makes this better than shellscripts

Some available modules:

  • command: run a specified shell command, default if nothing specified with -m
  • ping: does what it says and returns result
  • apt: debian/ubuntu package manager frontend
  • copy: copy local files -> remote hosts
  • file: change attributes
  • service: server daemon handling
  • template: create file from template and copy to remote host
  • lineinfile: make sure a certain line is found i certain config file

Run a shell command remotely:

## run uptime on host(s)
# -i: 'inventory', hostname or location of hosts file or a folder containing inventory files
# -m: 'module', ansible part that does something, defa
ansible -i HOSTNAME -m command -a uptime
ansible -i HOSTNAME_1,HOSTNAME_2 -a uptime

Group hosts:

## lets create a hosts file for less typing
cd ~
mkdir ansible
cd !$
echo "
[hosts-group]
HOSTNAME_1
HOSTNAME_2
" > hosts
# update hosts packages
ansible -i hosts -m date

Let's read the hosts file automatically be read and set explicitly some defaults. There's more to the hosts file like setting host/group-specific vars, groups-of-groups, but for now this will do.

echo "
[defaults]
host_key_checking = False
ansible_ssh_user = root
ansible_ssh_port = 22
ansible_connection = smart
ansible_shell_type = sh
hostfile = hosts
" > ansible.cfg
ansible hosts-group -a 'ip a'
ansible all -m ping  ## run on all hosts in the inventory file

-s does sudo, in case you'd want to work with a nonroot user.

more notation and some reasoning behind (proper) configuration management

All this can be done with bash, too, of course, except that you get some fancy output with colors and host grouping for free. But until now nothing really has changed. So why the fuss?

To better explain that, some more notation stuff:

  • A playbook consists of plays.
  • A play consists of tasks to run on (a) specific host(s).

And: Idempotence? The official definition from the manual's glossary:
An operation is idempotent if the result of performing it once is exactly the same as the result of performing it repeatedly without any intervening actions.

So:

  • The same results even if rerunning as often as you wish! (Want to implement errorhandling yourself in bash yourself for making sure you have a line in a file? Ansible may not do everything for you, but it helps rather nicely.)

But what is not expressed through this term, but what you get along the way compared to shell scripting?

  • host grouping (sure, go back to while read i'ing that files consisting lists of hosts.)
  • batteries included (no more thinking about handling connections or timeouts usually, modules for everything)
  • pretty fast time-to-production
  • error handling usually included, or providing you with enough readable info to work it out
  • automatically colored output while running
  • quite readable overview of what worked or didn't after running through
  • automatically documented systems if done properly

Especially the last two points make any maintenance work a breeze. Either recreate a configuration on a new host through provisioning. Or just quickly read the task lists (grouping them properly helps for sure here), to get the idea what the system's purpose was/is. Keep the last bullet point in mind when creating and structuring playbooks, it really pays off in the end. If I don't forget, I will include a virtual banana for scale or something, this post series is going to end up a long one anyway.

hating puppet

posted on 2017-04-29 10:20

a long rant on configuration management and a healthy dose of puppet hate

In the beginning there were shellscripts for provisioning. ('Go away or I will replace you with...')

Eventually highlevel language code appeared for this purpose. I did that myself with python, too reinventing the wheel, wrote some rather sophisticated wrappers in bash (shudder) for another project and have seen a nontrivial configuration management codebase in... PHP. I'm not kidding, that codebase still gives me the chills.

Due to the more or less obvious shortcomings with these methods, theoretically some proper configuration management tool, relieving one of the duty to really implement everything by hand while being readable, just had to arrive, especially with all these computers being everywhere nowadays.

From all the news and testimonals, puppet seemed to be the all-around best contender and thus lesser of all evils while being enterprisey enough.

Well puppet...

  • is just overhyped
  • has an awful documentation which looks much much better than it really is
  • which is making you end up regularily on github in the sources to find out what the heck its doing this time
  • because its internals do seem to change from time to time
  • along with questionable backwards compatibility (was also being told this by people working with it everyday with different versions in rather big different setups running in production.)
  • and it getting really ugly as soon as you try to do non-trivial parts (never try updating data structures with a loop),
  • but the other competition (chef, cfengine) is also just as bloated / crappy / ruby it seems / looks / people's testimonals tell,

... so you likely may up with ansible at last.

The fact that friends of mine jump through hoops with it at work in comparably big setups (and doing things with it in the process which the external specialized puppet consultants could not make do), also does not really inspire confidence. Mind you, they take the configuration management theme far, far more serious than I or my colleagues do. (The goal is single button presses for everything, from single hosts to complete clusters. The Litmus test wether the goal is achieved: You don't ssh onto the servers to do anything, but fix it in the configuration management's database. No more clustershell or for-looping across the fleets, no more manual work in the field. Same results, always, bla bla bla.)

And on the tool front, finally there's salt which a lot of people seem to dig, but ansible wins with simplicity, albeit being slower due to using ssh (salt has an encrypted rabbitmq as communication engine). But that hasn't been an issue with some several hundred servers so far, thanks to the ssh multiplexing.

But Salt basically needs a master due to being pull-based, but the push-based approach of ansible is good enough by far for provisioning with our workflow. Not every change has to go through ansible at our place (yet), and we just like having fewer moving parts.

On a last note, I really really really tried to dig (hoping even love) puppet, having had my doubts (scratch that, read: 'I still have') about ansible being the best tool available for our outlet, but I have looked too often at its documentation, just to end up finally in the sources to get whats happening. It doesn't matter that it is ruby instead of python (I am not one of THESE people), but It simply does not deliver, even after spending some considerable time with it. Sure, after a while you can even read all of its output pretty easily, but ansible's is just easier to read. You get up to speed rather easily, and until the ssh speed becomes the bottleneck, it will just do for now. May be that with 10.000s of servers this will change, lets see wether we will find out.

We've had our share with different ansible problems, too, of course (it's all software, after all). But all an all that went rather smoothly comparing the amounts of time we spent with fixing these. This included i.e. completely random bugs with extracting zipfiles and version incompatibilies, too, but all of them were resolved in an order of magnitude less pain and head-desk-banging happenings with puppet.

Dear future self, if considering puppet again, read this three times, thank you - and quit the stockholm syndrome bullshit.

puppet reference links and competition

posted on 2014-03-30 17:31:53

puppet is one of several open source configuration management tools.

A little excourse:
Notable alternatives would be Chef, Ansible, CFEngine, Salt and Sprinkle. In the Red Hat universe there exists Spacewalk, which also is built on puppet somehow according to wikipedia. A friend of mine considers Spacewalk a damn fine solution. Puppet seems to have an edge over its competition and is one of the bigger contestants along with Chef and CFEngine. Ansible and Salt are newer and likely better, Sprinkle is the smallest and most light-weight of the bunch. I haven't lookup up the current codebase sizes or commit activities which would indicate how lively each project is, this is as far as I came from searching blogs and test experiences.

As of end 2013 Ansible seems to be the most frictionless solution. Why is this about puppet then?

Puppet's unique selling points are

  • it works with Windows boxes, which is not the case with all other competitioners.
  • I might need it for work-related setups.
  • Spacewalk uses it internally, which speaks for its quality.

Back to puppet:
The type reference can be found here. This comes in handy when writing puppet manifest files. Its configuration reference can be found here. And for fun, the style-guide.

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apache2.4, apachebench, apple, applet, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, configuration management, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pct, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plasma, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pveam, pvecm, pvesm, pvresize, python, python3, qemu, qemu-img, qm, qmrestore, quicklisp, quickshare, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, ubuntu16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas