Posts tagged unit testing

Notes on unit testing

posted on 2013-10-24 11:34:10

Under the title Top 12 Selected Unit Testing Best Practices I found a list of notes on unit testing. Copy pasted them here and added some own notes.

1) Always Write Isolated Test Cases
- The order of execution has to be independent between test cases
JUnit can run tests in parallel IIRC, so this is a no-brainer. ALWAYS stick to this principle, else results will not be reproduceable. Non-reproduceability SI TEH EVIL, be it in building or testing.

2) Test One Thing Only in One Test Case
- Each test should focus on one aspect of an isolated method and/or class
This makes debugging easier and renders tests easier readable.

3) Use a Single Assert Method per Test Case
- Don't test too many different things at once! If you use just one Assert it's easier
This is the like 2), but goes further. Also growth of the test suite is directly visible in the amount of tests run. Senses of achievement are good for the programmers mind.

4) Use a Naming Convention for Test Cases
- A convention like: <Method-Name Under Test>_<Scenario>_<Expected-Outcome>
Readability and easier debugging through having information at a glance in the log... looks like a lot of doubled work, especially if tests are subject to change. But bear with it, it simply helps.

5) Use the AAA Pattern
- Use separated blocks ("ARRANGE", "ACT" and "ASSERT") help to improve maintainability
- Use comments to mark the blocks
I am not completely sure what this means. Maybe this relates to other unit testing frameworks than JUnit, where there are no explicit @Before or @After Task grouping mechanisms. If you have such a kind of approach, it could be helpful to document it in the header of your test file, even if you think your ideas are self-explanatory. If in doubt, document it, except it is likely to become obsolete again too soon.

6) Avoid the Expected Exception Tests
- Maybe @Test(expected = ArithmeticException.class) doesn't test what it claims to test
Expected failtests are implicit. Be explicit whenever possible.

7) Structure All Test Cases
- Like Short-/Long-Running, Integration-/Unit-tests, but don't use test suits to control order of execution
Order of execution or other depencenies are bad, same as 1).

8) Use Descriptive Messages in Assert Methods
- Describe the WHY and not the WHAT, not like Assert.assertEquals( a, b, "must be equal")
This is always key in documentation, be it in code documentation, changelogs or whereever. No matter in what language or what kind of code you are working on. Proofread stuff with this in mind.

9) Measure Code Coverage to Find Missing Test Cases
- Best indicator what is not tested, but don't be too sure that the code works
This helps for coming across parts of untested code, but may lead to false positives. 'Covered' code has tests written about it to check parts of itself, but this says nothing on the correctness of the tests by itself. Or, just as worse, not all test cases are covered by the test suite. Still it looks great for your boss, the higher the coverage key figure rises. :-)

10) Don't Forget to Refactor the Test Code
- Also maintain your test code
Basically there are like two types of refactoring. In short, one is reordering code within single method, which has little consequences. Ideal case for using tests. The other refactoring type is when you rebuild larger parts of software architecture, rendering your corresponding tests useless.

The more you think about proper design up front, the less refactoring of the second type is needed. Once huge parts of application code have to be refactored due to bad design, most of the time the amount of work, like, doubles when code is already fully tested... You get the idea. Some of the information sitting in the tests (domain knowledge) can be reused, but all tests need reworking. Here keep in mind, that 'perfect' is the enemy of 'good enough', and nobody gets paid for gold plating. If you are paid for gold-plating, that is a sign of deeper-rooting problems at work...

11) Limit Use of Mocks
- In some cases absolutely necessary, but with better design stubs should be enough
I have never thought about this. But I just realized I have to refresh my knowledge on Mocks vs. Stubs.

12) Use Parameterized Tests
- They can help avoid duplication and the best is the business gives you the data
For a lot of code to be tested, there are like three tests needed. This means one for the left border, one for the right border and one test for the data span in between. Of course, if there are more gaps in the domain of the data, more tests are needed. Anyways, parameterized tests are great if just your input data changes because they mean less work for you.

Sidenote: The better the specification, the easier it is writing correct tests. Basically better reqirements engineering up front makes test work easier afterwards.

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, 16.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, PS1, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apache2.4, apachebench, apple, applet, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, cmd, coleslaw, colorscheme, common lisp, configuration management, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fakeroot, fbsd, fdisk, fedora, file, files, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, gitbucket, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, make-jpkg, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pct, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plasma, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pveam, pvecm, pvesm, pvresize, python, python3, qemu, qemu-img, qm, qmrestore, quicklisp, quickshare, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, ubuntu16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas