Posts tagged dm-crypt

linux ssd and TRIM

posted on 2016-09-12 13:44

One of my setups is an encrypted LVM on top of a SSD.

The important part here is, should you want to TRIM the SSD, then TRIM has to be available on all layers:

  • SSD
  • dm-crypt
  • LVM

Now before you do anything, you might want to jump to the back of this post and to a test with fio for getting a baseline, so you got some numbers to compare for after you changed everything.

check storage media

hdparm, like always, is your friend:

root@ctr-014:~# hdparm -I /dev/sda | grep TRIM
    *   Data Set Management TRIM supported (limit 8 blocks)

If there is nothing to be found, TRIM just won't work for you.


grep discard /etc/crypttab:

sjas@host:~/blog$ grep discard /etc/crypttab 
sda5_crypt UUID=985f3826-e502-4402-ad01-5c13b84e9141 none luks,discard

If discard is not present, add it there.


grep issue_discards /etc/lvm/lvm.conf:

root@ctr-014:~# grep issue_discards /etc/lvm/lvm.conf
    #issue_discards = 0
    issue_discards = 1

Set it to "1" if it is not already.

update initramfs


update-initramfs -u


dracut -f


In case you changed anything above, to a restart so your settings become active.


for fs in $(lsblk -o MOUNTPOINT,DISC-MAX,FSTYPE | grep -E '^/.* [1-9]+.* ' | awk '{print $1}'); do fstrim "$fs"; done



You might want to put this into a file in /etc/cron.weekly with a #!/bin/sh shebang, to it will be run automatically.

test method

Test were done with fio:

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=512M --readwrite=randrw --rwmixread=75

Keep in mind these are single-threaded thus somewhat aenemic, but the result should become visible.


read iops = 19947
write iops =  6671


read iops = 3374 
write iops =  1128

actual measurement sample

root@ctr-014:/home/sjas# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=512M --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 512MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [78716KB/26344KB/0KB /s] [19.7K/6586/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3245: Mon Sep 12 13:40:44 2016
  read : io=392888KB, bw=79790KB/s, iops=19947, runt=  4924msec
  write: io=131400KB, bw=26686KB/s, iops=6671, runt=  4924msec
  cpu          : usr=5.77%, sys=33.72%, ctx=71337, majf=0, minf=6
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=98222/w=32850/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=392888KB, aggrb=79790KB/s, minb=79790KB/s, maxb=79790KB/s, mint=4924msec, maxt=4924msec
  WRITE: io=131400KB, aggrb=26685KB/s, minb=26685KB/s, maxb=26685KB/s, mint=4924msec, maxt=4924msec

Disk stats (read/write):
    dm-2: ios=95567/32093, merge=0/0, ticks=197872/140256, in_queue=355552, util=97.95%, aggrios=98222/33021, aggrmerge=0/0, aggrticks=203680/161852, aggrin_queue=366328, aggrutil=97.02%
    dm-0: ios=98222/33021, merge=0/0, ticks=203680/161852, in_queue=366328, util=97.02%, aggrios=96973/32744, aggrmerge=1249/277, aggrticks=197660/149212, aggrin_queue=347000, aggrutil=96.94%
  sda: ios=96973/32744, merge=1249/277, ticks=197660/149212, in_queue=347000, util=96.94%

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apachebench, apple, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pvecm, pvresize, python, qemu, qemu-img, qm, qmrestore, quicklisp, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh

Unless otherwise credited all material Creative Commons License by sjas